The prediction of in-hospital mortality in chronic kidney disease patients with coronary artery disease using machine learning models

Objective Chronic kidney disease (CKD) patients with coronary artery disease (CAD) in the intensive care unit (ICU) have higher in-hospital mortality and poorer prognosis than patients with either single condition. The objective of this study is to develop a novel model that can predict the in-hospital mortality of that kind of patient in the ICU using machine learning methods. Methods Data of CKD patients with CAD were extracted from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. Boruta algorithm was conducted for the feature selection process. Eight machine learning algorithms, such as logistic regression (LR), random forest (RF), Decision Tree, K-nearest neighbors (KNN), Gradient Boosting Decision Tree Machine (GBDT), Support Vector Machine (SVM), Neural Network (NN), and Extreme Gradient Boosting (XGBoost), were conducted to construct the predictive model for in-hospital mortality and performance was evaluated by average precision (AP) and area under the receiver operating characteristic curve (AUC). Shapley Additive Explanations (SHAP) algorithm was applied to explain the model visually. Moreover, data from the Telehealth Intensive Care Unit Collaborative Research Database (eICU-CRD) were acquired as an external validation set. Results 3590 and 1657 CKD patients with CAD were acquired from MIMIC-IV and eICU-CRD databases, respectively. A total of 78 variables were selected for the machine learning model development process. Comparatively, GBDT had the highest predictive performance according to the results of AUC (0.946) and AP (0.778). The SHAP method reveals the top 20 factors based on the importance ranking. In addition, GBDT had good predictive value and a certain degree of clinical value in the external validation according to the AUC (0.865), AP (0.672), decision curve analysis, and calibration curve. Conclusion Machine learning algorithms, especially GBDT, can be reliable tools for accurately predicting the in-hospital mortality risk for CKD patients with CAD in the ICU. This contributed to providing optimal resource allocation and reducing in-hospital mortality by tailoring precise management and implementation of early interventions. Supplementary Information The online version contains supplementary material available at 10.1186/s40001-023-00995-x.


Introduction
In the past few decades, chronic kidney disease (CKD) has become increasingly prevalent among various countries and regions around the world, increasing the enormous financial burden of many countries [1]. A major cause of death among patients with chronic kidney disease is cardiovascular disease [2], and CKD patients with coronary artery disease (CAD) have a poorer prognosis than CKD patients without CAD [3,4]. Moreover, the risk factors of patients with CKD combined with CAD are much different from those with only CAD [5]. Some studies demonstrated that atherosclerosis is the leading cause of death in advanced CKD patients with CAD, especially end-stage renal disease (ESRD) patients [6]. In addition, the pathogenesis of CKD patients with CAD has not been clearly elucidated [7]. Thus, the present indicators and prediction models perform poorly in predicting clinical outcomes for CKD patients with CAD.
Machine learning (ML) is a cutting-edge technology with the rapid development of artificial intelligence [8]. Compared to the traditional statistical method, ML has better clinical predictive accuracy and performance with faster processing speed [9]. With the development of the online public standard database, such as the Medical Information Mart for Intensive Care IV (MIMIC-IV), ML has increasingly penetrated the medical analysis field [10]. However, a few ML algorithms focused on the mortality prediction of CKD patients with CAD.
The purpose of our study is to (1) construct novel predictive models based on the various machine learning algorithm for in-hospital mortality of patients with CAD and CKD in intensive care units (ICU); (2) select an ML model with the best predictive performance and clinical value; and (3) validate these ML models via external set from the Telehealth Intensive Care Unit Collaborative Research Database (eICU-CRD) database.

Data sources
Data from the MIMIC-IV database were used in this study to establish predictive models for patients with CKD and CAD [11]. MIMIC-IV was a free, online accessible public database containing more than 50,000 ICU admissions from 2008 to 2019 in Beth Israel Deaconess Medical Center (Boston, Massachusetts). Data from eICU-CRD were used as an external validation cohort [12]. Over 200,000 ICU admissions from 208 hospitals across the country were compiled in the eICU-CRD, which was a publicly available multicenter database. The MIMIC-IV and the eICU-CRD database included the following information: demographics, vital signs, laboratory results, and diagnosis of International Classification of Diseases and Ninth Revision (ICD-9) codes. One author (ASY) obtained the certification to access these databases and extracted variables needed in the study (certification number: 39674606). Patients in these databases were unidentified with their health information, so individual patient consent was not required.

Study population and data extraction
All patients diagnosed with CAD and CKD were included in this study. Patients who stayed in ICU for less than 6 h, less than 18 years old, without baseline creatinine results, and with missing data > 30% were excluded. Only the first admission was taken into account if a patient had multiple admissions. Baseline creatinine was defined as the creatinine level in the patient's first blood test after hospital admission. Data of demographic information, lab results, hourly vital signs, comorbidities, medications (including aspirin, clopidogrel, ticagrelor, statin, betablocker, NOAC, and warfarin), operative procedures, ICU stay details, and in-hospital mortality were extracted from MIMIC-IV and eICU-CRD database using pgAdmin PostgreSQL tools (version 1.22.1).

Data preprocessing and feature selection
Variables with > 30% missing values were dropped, and multiple imputations were conducted for other vacant data. Multivariate Imputation by Chained Equations (MICE) was performed and returned an object containing five complete datasets. Then, statistical models such as linear regression or generalized linear model were applied to each complete dataset in turn for interpolation modeling. The pool function consolidates these individual analysis results into a group. The complete dataset is finally returned based on the standard errors and P-values of the model. MIMIC-IV and eICU (external validation data) databases were imputed separately using the fully conditional specification to avoid data leakage via the "mice" package in R [13].
Feature selection was a crucial process of reducing the number of features in a massive dataset according to the importance of the study variables. The Boruta algorithm was a wrapper method for feature selection built around the Random Forest Classifier algorithm. During the model construction, Boruta created a copy of the original dataset features as Shadow Features and compared the Z-score between the actual features and shadow features calculated via Random Forest Classifier in each iteration. If the Z-score of an actual feature was higher than the maximum Z-score of shadow features, this feature was considered pivotal and kept; otherwise, it was dropped [14].

Statistical analysis
Patients were divided into two groups according to whether they survived to discharge. Categorical variables were summarized as numbers with percentages and compared by Fisher's exact probability method (or Chi-square tests). The Wilcoxon rank sum test was used to test continuous variables that were expressed as the median with interquartile ranges.
Eight machine learning models, including logistic regression (LR), random forest (RF), Decision Tree, K-nearest neighbors (KNN), Gradient Boosting Decision Tree Machine (GBDT), Support Vector Machine (SVM), Neural Network (NN), and Extreme Gradient Boosting (XGBoost), were established to develop the predictive models. 70% of the patients from MIMIC-IV were randomly extracted as the training set, while the remaining 30% was utilized for internal validation. Tenfold crossvalidation was performed in each model to prevent overfitting to acquire average accuracy. The performance of each model was evaluated by the area under the receiver operating characteristic (ROC) curve (AUC) and average precision (AP) from precision/recall (P-R) curves in the validation set. Further, the model with the best performance was picked up to recognize the risk factors most related to in-hospital deaths interpreted by Shapley Additive Explanations (SHAP) method. The SHAP value visually exhibited each feature's importance and contribution to in-hospital mortality. In addition, data from eICU-CRD were used as external validation to assess the prediction model's performance. Decision curve analysis (DCA), AUC, and calibration curves were conducted to evaluate the clinical application and the consistency of the predictive probabilities.
All statistical analyses, machine learning algorithms, and SHAP were implemented via Python (version 3.9.12). The Boruta algorithms were conducted by R (version 4.1.3, Austria). A P-value lower than 0.05 (two-sided) was regarded as statistically significant.

Baseline characteristics
A total of 3590 CKD patients with CAD from MIMIC-IV and 1657 CKD patients with CAD from eICU-CRD were included in this study cohort according to the inclusion and exclusion criteria. Figure 1 Tables 1, 2.
Patients who died during the hospitalization have higher serum creatinine and troponin level and higher myocardial infarction, heart failure, and arrhythmia risks (P < 0.001).

Feature selection
According to the Boruta algorithm analysis, 76 of 124 variables most closely associated with in-hospital mortality were selected (Fig. 2). Based on the Z-values, the top twenty variables are the history of cardiac arrest, sequential organ failure assessment (SOFA) score, the maximum values of aspartate aminotransferase (AST) and phosphate, the average values of spo2, white blood cell (WBC), AST, systolic blood pressure (sbp), sodium and platelet, and the minimum values of oxyhemoglobin saturation (spo2), SBP, heart rate, WBC, AST, glucose, phosphate, partial thromboplastin time (PTT), and mean blood pressure (mbp). Although the Z-values for acute coronary syndromes and diabetes were lower than the maximum Z-value of shadow feature, they were included in the analyses based on clinical experience. Therefore, a total of 78 variables were selected for the machine learning model development process.

Machine learning model development and comparisons
Eight machine learning models were generated to predict the in-hospital mortality in CKD patients with CAD. Among the eight models, GBDT had the best predictive value of in-hospital death, with AUC = 0.946 and AP = 0.778. Figure 3 exhibited the discrimination performance of these machine learning models via ROC and P-R curves after ten cross-fold-validation in the test set. The SVM (AUC = 0.937), XGBOOST (AUC = 0.939), and GBDT had superior performance in the predictive ability for in-hospital death of CKD patients with CAD compared to the traditional logistic regression model. A set of detailed performance metrics for various machine learning models is presented in Table 2.

Visualization by SHAP
The SHAP algorithm was conducted to visually exhibit each factor's importance to the hospital mortality predicted by the GBDT model. Figure 4A shows the feature importance plot, including 20 significant variables most correlated to in-hospital death in descending order. The age factor had the most potent predictive power, followed by the minimum value of spo2 and warfarin. Figure 4B presents whether that feature is high (in red) or low (in blue) for that observation according to the SHAP value.
The utilization of warfarin has a negative impact on inhospital mortality.

Subgroup analysis
Subgroup analyses were conducted stratifying by ACS and dialysis condition. Age was no longer the most potent predictive factor in ACS and non-ACS patients and warfarin dropped out of the top 20 significant variables in ACS patients. SOFA score had the most potent predictive value in dialysis patients followed by glucose level. Interestingly, phosphate level was one of the top 20 influencing factors in non-dialysis patients, but its predictive value in dialysis patients was limited (Additional file 1: Fig. S1, Additional file 2: Fig. S2).

External validation
A total of 1657 CKD patients with CAD were extracted from the eICU-CRD database as an external validation dataset to verify the predictive accuracy of the selected GBDT model. Additional file 3: Table S1 exhibits the baseline characteristics of these patients. A total of 211 (12.7%) patients died during hospitalization. Taken together, GBDT had good predictive values (AUC = 0.865, AP = 0.672), while the clinical value was limited in the validation cohort based on the result of DCA and calibration curve (Fig. 5).

Discussion
Patients with CKD and CAD became more and more popular in recent decades. And mortality in patients suffering from these two conditions is twice as compared to patients with CAD alone [4]. Despite the increased incidence and incredibly lethal, their patients were excluded from most clinical trials due to the disease complexity and treatment conflicts. To date, factors associated with the prognosis in CKD patients with CAD were not clear and current risk stratification tools could not be applied to these patients. With the development of artificial intelligence, accurate prediction of these complex conditions could be achieved using machine learning methods. MIMIC-IV and eICU-CRD were large-scale and highquality databases performed in many crucial pieces of research in recent years. In this retrospective study, CKD patients with CAD admitted to ICU were extracted from MIMIC-IV to develop predictive models for in-hospital mortality via various ML algorithms. The GBDT model outperformed the predictive performance of seven other ML algorithms, including LR, RF, Decision Tree, KNN, SVM, NN, and XGBoost, according to the features selected by the Boruta algorithm. Next, the SHAP method was conducted to explain GBDT visually, ensuring clinical interpretability and facilitating the utilization of the prediction model. The performance and clinical application value of GBDT were also validated by an external set from the eICU-CRD database. This is the first prediction method especially for CKD patients with CAD to evaluate the in-hospital mortality with precise efficiency in two large cohorts, which means good generalization to extend to clinical practice. Depending on the visualization technique SHAP, our study identified several crucial variables related to the in-hospital mortality of patients with CKD and CAD in the ICU. This study identified a factor strongly associated with the in-hospital mortality observed in our study which was serum phosphate. Previous studies have shown that elevated serum inorganic phosphorous (P) is tightly associated with cardiac death in CKD patients [15]. A national study illustrated that hyperphosphatemia could lead to a predisposition to metastatic calcification and the development and progression of secondary hyperparathyroidism, which may contribute to the abundant morbidity and mortality of patients with ESRD [16]. Another research with a 2-year follow-up also identified strong relationships between hyperphosphatemia and cardiac causes of death in hemodialysis patients [17]. Moreover, a cross-sectional study showed elevated serum levels of P were significantly related to calcified coronary atherosclerotic plaque detected by cardiac computed tomography, even in patients with normal kidney function [18]. The previous studies exhibited the significance of P in prognosis in CKD patients. In our study, we focused on CKD patients with CAD and showed that serum P was a strong predictor of in-hospital mortality.  [27]. In addition, Pezel and colleagues developed multiple fractional polynomial algorithm ML models, including 31,752 consecutive patients, to predict 10-year death [28]. This ML model also has a higher prognostic value than traditional clinical or Cardiac Magnetic Resonance scores (AUC 0.76). However, the mechanism of CKD combined with CAD is more complex and harder to explain than the mechanism of CAD alone [4]. For example, statin lipid-lowering therapy is still contradictory in improving the prognosis of patients with ESRD and CAD [29]. Predictions based on the traditional model cannot be made with reasonable accuracy  and comprehensiveness for patients suffering from such complex diseases [5,30]. For this reason, machine learning is of great significance. The GBDT algorithm, also known as the multiple additive regression trees, has more accurate predictive ability and sophisticated algorithms than the LR, decision tree, and random forest algorithms [31]. It has many nonlinear transformations and solid, expressive ability, and does not require complex feature engineering and transformation [32]. The XGBoost model, a modified GBDT algorithm, could cope efficiently and flexibly with missing data and combines weak predictors to produce accurate predictions [33]. The no free lunch theorem (NFL) illustrates that the expected performance of each learning algorithm is the same if all possible problems are considered, which means there is no single, universal best machine learning algorithm for every situation [34]. Among eight ML models, the GBDT model performed the best clinical predictive value in in-hospital mortality risks in this kind of patient.
The advantages of this study were that it was the first study focusing on the in-hospital mortality for CKD patients with CAD in ICU based on a public database and constructed an ML model to predict it with external validation. Some limitations must be acknowledged. First, MIMIC-IV was a single-center database; most white patients may lead to racial bias and limit the applicability to other populations. However, external validation was applied using data from a multicenter database, eICU-CRD. Second, the deviation of missing data was inevitable because the data were extracted from the open public database. We performed fully conditional specification (FCS) implemented by the MICE algorithm to multiply and impute the missing data. Third, the selection bias was inevitable because this was a retrospective and observative study. Data were extracted from two different databases as internal and external sets, and further multicenter and large-scale clinical research was still needed. Nevertheless, the constructed ML model still may contribute to clinicians improving the prognosis and treating CKD patients with CAD at high risk in ICU timely. Collecting clinical data on ICU patients have been difficult due to the impact of the CoronaVirusDis-ease2019 outbreak. Public databases have helped tide clinical workers over worldwide. But more prospective multicenter clinical studies should also be established for further research.

Conclusions
In conclusion, machine learning algorithms can be reliable tools for accurately predicting the in-hospital mortality risk for CKD patients with CAD in the ICU. GBDT technology had the best predictive performance, which may provide optimal resource allocation and reduce inhospital mortality by tailoring precise management and implementing early interventions.